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In 0.9 fb _1 of pp collisions, DO has observed an excess of events with an isolated lepton, missing 
transverse momentum, and two to four jets. This excess is consistent with single top quark 
production. We examine these data to study the Lorentz structure of the Wtb coupling. The 
standard model predicts a left-handed vector coupling at the Wtb vertex. The most general lowest 
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Recently, we presented evidence for single top quark production in pp collisions at y/s = 1.96 TeV [l| based 



on 0.9 fb _1 of data collected using the DO detector [2j 
at the Fermilab Tevatron collider. In this Letter, we 
report an extension of this analysis using the same data 
set and similar analysis tools to study the consistency 
of this excess with different hypotheses for the couplings 
involved in single top quark production. This is the first 
time such a test has been carried out. 

The standard model (SM) has been extraordinarily 
successful in describing the data taken at the energies of 
present colliders. However, we know that the electroweak 
symmetry breaking sector of the SM gives rise to many 
unanswered questions, making a strong case for new 
physics beyond the SM. This new physics can manifest 
itself in the production of new particles or in corrections 
to SM processes that change the effective couplings of 
SM particles. The interactions between quarks and gauge 
bosons have been measured precisely at the CERN Large 
Electron Positron collider [3(] except for the top quark, 
which was not kincmatically accessible. The large mass 
of the top quark has prompted speculation that the 
top quark may play a special role in the mechanism 
of electroweak symmetry breaking and thus have non- 
standard interactions with weak gauge bosons. We can 
probe the interactions of top quarks with W bosons via 
measurements of single top quark production and top 
quark decays in tt production, each yielding comple- 
mentary information. 

The dominant tree level Feynman diagrams for single 
top quark production in pp collisions are illustrated in 
Fig. Q] We use the notation "tb" for the sum of the 
s-channel processes tb and tb and "tqb" for the sum of 
the t-channel processes tqb and tqb. We assume that 
single top quark production proceeds exclusively through 
W boson exchange. Therefore, extensions of the SM in 
which single top quarks are produced via flavor-changing 
neutral current interactions [4J or the exchange of new 
massive scalar |5j or vector bosons [6| , are not considered 
here. We further assume that \V t d\ 2 + \V ts \ 2 < \Vtb\ 2 , 
i.e., the Wtb vertex dominates top quark production and 
decay. Finally, we assume that the Wtb vertex is CP 
conserving. 

The most general, lowest dimension, CP conserving, 
Lagrangian for the Wtb vertex is [7| : 
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where Mw is the mass of W boson, Pj, = (1 — 7s)/2 is 
the left-handed projection operator and Pr = (1 + 7s)/2 
is the right-handed projection operator. In the SM the 
values of the form factors are ff 1 « 1, f£ = //* = f 2 = 
0. In this case the predicted cross section for single top 
quark production is 2.9 ± 0.3 pb [8(. 

The presence of anomalous couplings can change 
angular distributions and event kinematics as 



(a) 



(b) 





FIG. 1: Feynman diagrams for (a) s-channel and (b) t-channel 
single top quark production. 



demonstrated by the pr spectrum of the charged 
lepton from the decay of the top quark in Fig. [2] 
Such differences can be used to distinguish these 
couplings @, [lfj. The magnitude of the right-handed 
vector coupling and tensor couplings can be indirectly 
constrained by the measurement of the b — > sj branching 
fraction [llj. Direct constraints on the combination of 
several couplings can be obtained from the measurement 
of the W boson helicity in top quark decays [12j . The 
predicted single top quark production cross sections are 
2.7 ± 0.3 pb if /f = 1 and 10.4 ± 1.4 pb if / 2 L = 1 or 
f 2 = 1 and the other couplings vanish [9j . 

Ideally, we would like to set limits on all four couplings, 
f\i fi ' ■> fi' an d f 2 ■ simultaneously. This, however, 
requires more data than is currently available. We 
therefore look at two couplings at a time and assume 
that the other two are negligible. We consider three 
cases in which we allow the left-handed vector coupling 
fi and any one of the three non-standard couplings to 
be non-zero. We refer to these as (Li, L 2 ), (£i, i?i), and 
(L U R 2 ). 

We look for events in which the top quark decays to 
a W boson and a b quark, followed by the decay of the 
W boson to an electron or a muon, and a neutrino. The 
event selection is the same as in Ref . [lj . To enhance the 
signal content of the selected data sample, one or two of 
the jets are required to be identified as originating from 
long-lived b hadrons [131 ]. 

We model the single top quark signal using the 
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COMPHEP-SINGLETOP Monte Carlo event generator 
and the anomalous Wtb couplings arc considered in 
both production and decay in the generated signal 
samples. The event kinematics for both s-channel and 
t-channel reproduce distributions from next-to-lcading- 
order calculations [8| . The decay of the top quark and 
the resulting W boson are carried out in the SINGLETOP 
generator in order to preserve the information about the 



spin of the particles, pythia [15j is used to add the 
underlying event, initial and final-state radiation, and 
for hadronization. The top quark mass is set to 175 GeV 
and the CTEQ6L1 parton distribution functions [16| are 
used. 

Background contributions from VF+jets and tt 
production are simulated using the ALPGEN leading- 
order Monte Carlo event generator 17J interfaced to 



pythia. A parton-jet matching algorithm [18[ is used 
to avoid double counting. The response of the DO 
detector to the Monte Carlo events is simulated using 
GEANT [20|. Simulated events are processed through 
the same reconstruction software used for data and 
efficiencies and resolutions are corrected to match the 
performance of the reconstruction for data. The tt 
background is normalized using the theoretical cross 
section [l9j. The multijet background is modeled using 
events from data containing nonisolated leptons that 
otherwise resemble the signal events. The W±jets 
background is normalized such that the number of events 
predicted by the simulation agrees with the number of 
events observed in each analysis channel (defined by 
lepton flavor and jet multiplicity) before b tagging is 
applied. 

After all cuts we select 1,398 b tagged lepton+jets 
events, which we expect to contain 62 ± 13 single top 
quark events, 348±80 tt events, 849±222 W+jets events, 
and 202 ± 48 multijet events. Within each channel the 
signal efficiency of the complete selection does not depend 
strongly on the assumed Wtb coupling. The selection 
efficiencies for signal with different Wtb couplings vary 
between (1.07 ± 0.15)% and (1.52 ± 0.16)% for tb events 
with 1 b tag, between (0.86 ±0.13)% and (1.14 ±0.14)% 
for tqb events with 1 b tag, between (0.40 ± 0.08)% and 
(0.60 ± 0.10)% for tb events with 2 b tags, and between 
(0.07 ± 0.01)% and (0.10 ± 0.02)% for tqb events with 2 
b tags. 

Systematic uncertainties in the signal and background 
models are estimated using the methods described in 
Ref. [l| . The dominant contributions to the uncertainties 
in the background estimate come from: the normalization 
of the tt background (18%), which includes the top 
quark mass uncertainty; the normalization of the PF±jets 
and multijets backgrounds to data (17%-27%), which 
includes the uncertainty in the fraction of events with 
heavy flavor production; and the 6-tagging efficiencies 
(12%-17% for double-tagged events). The uncertainties 
from the jet energy scale corrections (l%-20%) and 
the b tagging probabilities affect both the shape and 
normalization of the simulated distributions. All other 
components contribute at the few percent level. 

We use boosted decision trees [2l|, |22[ to discriminate 
between the single top quark signal and background. For 
each of the three coupling scenarios, we train trees in 
four analysis channels defined by lepton flavor and b tag 
multiplicity. For each scenario the signal samples consist 
of a sample of events generated with left-handed vector 
coupling set to one, i.e. with SM coupling, and a sample 
of events generated with the non-standard coupling set to 
one and all other couplings set to zero. The background 
sample consists of events from all background sources in 
the expected proportions. 

We use 50 variables in the training, the 49 variables 
that were used in Ref. [l| plus the lepton px which helps 
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FIG. 2: Charged lepton pr spectrum from data and 
expectation for SM single top production plus background for 
events with two jets, one fc-tagged jet. Superimposed are the 
distributions from single top quark production with different 
couplings (all other couplings set to zero) normalized to ten 
times the SM single top quark cross section. 



distinguish the signals with different couplings, as can 
be seen in Fig. O The variables describe individual 
object kinematics, global event kinematics, and angular 
correlations. The boosted decision trees produce a 
continuous output distribution ranging from zero to one, 
with background tending closer to zero and signal tending 
closer to one. Figure [3] shows representative output 
distributions for the data and the sum of SM signal and 
backgrounds for the electron channel with two jets and 
one 6-tagged jet in each of the three anomalous coupling 
scenarios. 



We use Bayesian statistics [23J to compare the 
output distribution of the decision trees from data to 
expectations for single top quark production. For any 
pair of values of the two couplings that are considered 
non-zero, we compute the expected output distribution 
by superimposing the distributions from the two signal 
samples with the non-standard coupling and from the 
background samples in the appropriate proportions. 
In case of the (Li,L2) scenario, the two amplitudes 
interfere, and we use a superposition of three signal 
samples, one with left-handed vector couplings, one 
with the left-handed tensor coupling only set to one, 
and one with both couplings set to one to take into 
account the effect of the interference. We then compute 
a likelihood as a product over all bins and channels. 
Here we use twelve channels defined by lepton flavor, 
b tag multiplicity, and jet multiplicity (2, 3, or 4). We 
assume Poisson distributions for the observed counts, and 
flat nonnegative prior probabilities for the signal cross 
sections. The prior for the combined signal acceptance 
and background yields is a multivariate Gaussian with 
uncertainties and correlations described by a covariance 
matrix. A two-dimensional posterior probability density 
is computed as a function of \f[\ 2 and |/x| 2 , where 
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FIG. 3: Boosted decision tree output distributions for data and sum of SM signal and backgrounds for events with two jets 
and one 6-tagged jet for (a) the [L\,L2) scenario, (b) the (L\,R\) scenario, and (c) the (Li,R,2) scenario. Superimposed are the 
distributions for the single top quark signals with different couplings normalized to five times the SM single top quark cross 
section. 
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FIG. 4: Plots of the two-dimensional posterior probability 
density for the anomalous couplings. The plots on the left 
show the expectation for left-handed vector couplings and the 
plots on the right show the observed posterior from our data. 
The upper row (a, b) shows the plots for the [L\,Li) scenario, 
the middle row (c, d) for the (L\,Ri) scenario, and the bottom 
row (e, f) for the (Li,R.2) scenario. 



fx is any of the other three non-standard couplings, 
in each channel. These probability distributions arc 
shown in Fig. 2] We quote the values of the couplings 



TABLE I: Measured values of the total cross section for single 
top production and one-dimensional limits on Wtb couplings 
in the three scenarios. 



Scenario 



Cross Section 



(Li,L 2 ) 



AAtVi pb 



TTT 



5.2 



-2.5 
-2.6 



H2.2 
-2.2 



P b 



I/: 
Itfl 



\ft\ 



Coupling 

- I 4 +U.« 

— lA -o.b 

< 0.5 at 95% CL. 

= i-s±i;g 

< 2.5 at 95% CL. 

- 1 4+ ' 9 

< 0.3 at 95% CL. 



that maximize the two-dimensional likelihood as our 
measurements. In all three scenarios we measure zero 
for the right-handed vector, and left- and right-handed 
tensor couplings. We compute 95% CL. upper limits on 
these couplings by integrating out the left-handed vector 
coupling to get a one-dimensional posterior probability 
density. The measured values are given in Table |TJ The 
data favor the left-handed vector hypothesis over the 
alternative hypotheses. 

In summary, we have studied the excess observed 
in 0.9 fb _1 of DO data in the search for single top 
quark production. We attribute this excess to single top 
quark production and study its consistency with different 
hypotheses for the structure of the Wtb coupling and 
find that the data prefer the left-handed vector coupling 
over the alternative hypotheses studied. These are the 
first direct constraints on a general Wtb interaction and 
the first direct limits on left- and right-handed tensor 
couplings. 
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